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Abstract: The continuous monitoring of movements using artificial intelligence technique has become a 
necessary system to cob the level crossing problems faced around the railway infrastructure. This project 
work has developed a model for detecting movements using image processing method with convolutional 
neural network, an artificial intelligence technique in monitoring and detection of movements in the 
railway infrastructure. Images containing movements and non-movements in railway infrastructure were 
acquired from Google image search have been used in the implementation of the system and a deep 
convolutional neural network model has been developed. The proposed model was able to classify 11 out 
of 12 images correctly in the evaluation of the train dataset and the performance accuracy of the model 
recorded was 92% with a loss of 0.4% which resulted to the misclassification of one of the images. The 
analysis of the system showed that the system can be optimized to perform better. 
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Introduction 


The railway system of transportation in Nigeria has experienced a significant improvement since the 
Federal government began to look into that direction in 2015. Directing their attention towards that aspect 
of transportation has in recent times boosted even the economy. Before then, the railway infrastructure 
suffered decay and dilapidation from the vibrant railway system that was handed over to us by the 
colonial masters (Oraegbune & Ugwu, 2020). Infrastructure and the quality of it, is a pillar when it comes 
to competitiveness in the railway system (Bank, 2017). However, to improve on the quality and safety of 
the rail system, there is need to monitor movements on the railways. In Ristic-Durrant et al. (2021) paper, 
they opined that about 442 accidents that occurred and were reported in the European Union were as a 
result of level crossing accidents; this includes pedestrians. Therefore, identifying and detecting 
movement on time in railway infrastructure is very important for the safety of the railway system 
(Andrusca et al., 2020). 


In recent times, the experimental researches carried out by technologists and researchers with particular 
reference to Artificial Intelligence (AI), Internet of Things (loTs) and Smart Systems has drastically 
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improved the way our environment is perceived. The use of the sensor technology in the development of 
continuous monitoring systems especially to obstacle detection in road transportation has been prominent 
in published works (Ristic-Durrant et al., 2021). The authors Hromadka et al. (2020) has pointed out the 
need to put preventive measures on our railway infrastructures in other to ensure railway safety and 
reduce the number of occurrences. This can be implemented with the help of Artificial Intelligence 
techniques; which will pave way for real time assessment of railways and detection of any form of 
movement. In heavy traffic areas, it is also very important to have movement intention recognition system 
to checkmate movements by humans and pedestrians which is an aspect of computer vision, image 
processing and pattern recognition. The system uses the artificial intelligence method in the detection of 
movement around the railway infrastructure. 


Artificial Intelligence has become very significant in our everyday life and machine learning applications 
especially in image classification is now an important aspect in technological explorations. Convolutional 
Neural Network (CNN) has been known to be used extensively in image and video classification and 
analysis. It has the capability of detecting important features with no intervention of humans (Sarker, 
2021). CNN uses the integration of more properties and not the regular multiplication of weights and 
neuron outputs (Wang & Xi, 1997). CNN has been used for many applications such as image 
enhancement, feature extraction, computer vision, medical imaging and the likes. It has the learning 
ability to detect different features from an input image applying efficient computational methods (Park et 
al., 2021). The motivation in the design of CNN is due to the discovery of visual mechanism of the brain 
known as the visual cortex which contains a lot of cells responsible for the detection of light in the visual 
field known as receptive fields (Hijazi et al., 2015). This research work is exploring the advantage of 
convolutional neural network in the detection of movement in railway infrastructures. Creating a model 
for detecting movements in railway infrastructure has been necessitated by the occurrences recorded. 


According to Ristic-Durrant et al. (2021), about 1666 railway accidents were recorded in 2018 by the 
European Union. The causes of such accident were: level crossing, rolling stock in motion among others. 
The non-artificial detection of such movement (level crossing, rolling stock in motion) on the railway 
infrastructure is the problem statement of this article. Artificial detection of movement will help reduce 
such casualties experienced in the railways which will in-turn reduce accident. Thereby, improving the 
safety experience in the railway infrastructure. The proposed aim is the creation of a model for detecting 
movement in railway infrastructure using Convolutional Neural Network in Image Processing. The 
system is developed following the outlined objectives: 1) To identify the targeted movements that is being 
observed on the railway infrastructure. 2) To create a model framework for the detection of movement in 
railway infrastructure. 3) Acquire, pre-process and encode the images to enable the extraction of features 
in R programming language and 4) to perform classification and detection analysis applying the CNN 
model on the acquired dataset. 


Related Literature 
Theoretical Framework 


Kovacevic et al. (2018) gave a report on the use of remote monitoring for slope stability assessments in 
infrastructure. Traditionally, railway lines have been considered as low-technology in terms of transport 
infrastructure. However, the railways of today are accommodating ever increasing speeds and are 
operating at ever higher levels of efficiency. Soni (2016), in his thesis, presented a non-contact monitoring 
of railway infrastructure with terrestrial laser scanning and photogrammetry at network rail. Current 
monitoring practices in the railway industry primarily rely on total station and prism-based methods. 
Jhapate et al. (2020) present a paper on unusual crowd activity detection using OpenCV and motion 
influence map. Suspicious behaviour is dangerous in public areas that may cause heavy causalities. 
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Vadlamudi (2020) performed an evaluation of object tracking using OpenCV in Python. Object Tracking 
System used to track the motion trajectory of an object in a video. Irianto et al. (2009) presented motion 
detection system using OpenCV library. In this decade, the need of monitoring systems in every field is 
increasing rapidly. Monitoring systems are applied to improve the security systems and the productivities. 
Dewangan & Chouhan (2020) did a review on object detection with OpenCV. Due to object detection’s 
close relationship with video analysis and image understanding, it has attracted much research attention in 
recent years. Ravish Aradhya (2019) presented an object detection and tracking with deep learning and 
artificial intelligence for video surveillance. 


Review of Empirical Studies 


Oraegbune & Ugwu (2020) presented a paper on delivering sustainable transport infrastructure projects 
with railways in Nigeria as the focus. This study presented the issue of railway transportation 
infrastructure sustainability which involves multi-dimensional view of sustainability criteria such as 
economy, environment and society with the problem of non-systematic method of assessment in Nigeria. 
Andrade (2008) in his paper renewal decisions from a life-cycle cost (LCC) perspective in railway 
infrastructure presented an integrative approach using separate LCC models for rail and ballast 
components. Stenstrom (2014) presented a model and methods of operation and maintenance performance 
of rail infrastructure. Congestion of roads and sky, increasing energy costs and a demand to reduce 
emissions have created a need to shift transportation from road and air to rail. Hromádka et al. (2020) 
assesses socioeconomic aspect of railway infrastructure project life cycle and Liden (2015) presented a 
survey of planning problems and conducted research on railway infrastructure maintenance that consumes 
large budget. Akwetteh et al. (2021) looked into the current railway development and its influencing 
factors in Ghana. Doll et al. (2015) in their paper looked at the results and efficiency of railway 
infrastructure financing with the EU. 


Review of Related Literature 


Andrusca et al. (2020) presented a condition monitoring system and fault detection for impedance and 
bonds from railway infrastructure. They opined that, nowadays, sensors and condition monitoring systems 
are expanding rapidly and becoming cheaper. Risti¢-Durrant et al. (2021) did a review of vision-based 
On-Board obstacle detection and distance estimation in railways. This paper provides a review of the 
literature on vision-based on-board obstacle detection and distance estimation in railways. Arastounia 
(2015) has developed an automated recognition of railroad infrastructure in rural areas from LIDAR data. 
This study is aimed at developing automated methods to recognize railroad infrastructure from 3D 
LIDAR data. A laser-based obstacle detection at railway level crossings has been presented by these 
authors. This paper presents a system for obstacle detection in railway level crossings from 3D point 
clouds acquired with tilting 2D laser scanners (Amaral et al., 2016). Chang et al. (2014) has developed 
railway infrastructure monitoring system with satellite radar data. The feasibility of monitoring the 
stability of railway infrastructure using dedicated satellite radar observations is demonstrated. 
Perspectives on railway track geometry condition monitoring from in-service railway vehicles was carried 
out by these authors. Their paper presents a view of the current state of monitoring track geometry 
condition from in-service vehicles (Weston et al., 2015). Elberinka et al. (2013) in their paper rail track 
detection and modelling in mobile laser scanner data has presented a method for detecting and modelling 
rails in mobile laser scanner data. Sahebdivani et al. (2020) has presented rail track detection system and 
projection-based 3D modelling from UAV point cloud. Sattar et al. (2018) presented a review on road 
surface monitoring with smartphone sensors. Road surface monitoring is a key factor to providing smooth 
and safe road infrastructure to road users. Parven & Shah (2021) presented a motion detection system 
using OpenCV in Python. This paper suggests a motion detection software system that enables us to see 
the movement around an object or a visual area. 
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System Analysis 


The proposed system presents movement detection model in the railway infrastructure. This is achieved 
by creating a model employing machine learning tools to monitor movement around the railway 
infrastructure. A Convolutional Neural Network algorithm has been employed using image data acquired 
from Google image search database. The CNN algorithm learns from the data and is able to classify and 
detect such movement when seen in a new captured data and with that, an alarm/ alert is sounded to notify 
the appropriate security operatives for prompt action. This model is represented in the Figure | below. 
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Figure 1: Architectural Model of the proposed Movement Detection in Railway Infrastructure 


The architectural model as shown in Figure 3.1 explains a model for detecting movements in railway 
infrastructure using convolutional neural network in image processing. The model shows the data 
acquisition and pre-processing, CNN Detection model development and the implementation analysis of 
the proposed system. The implementation of this model helps security operatives not to miss any form of 
movement around the railway infrastructure thereby preventing unforeseen casualties that may arise. 
Further explanations on the proposed model is detailed as followed: 


A. Data Acquisition and Pre-processing 


The system requires data for the model to be developed. An image dataset was acquired through Google 
image search and is uploaded to the work environment of the system. The images, during acquisition 
comes in different sizes. Hence, the images were resized to 100 x 100 and the parameters extracted, re- 
organised for use by the model and encoded with binary “0” and “1” for implementation. The acquire 
dataset comes in 3 channel (RGB), meaning, it is a coloured image. The total image used for the 
implementation demonstration is 20 with 8 of the images used as test image and a total of 12 images used 
for training the model. 


B. CNN Detection Model Development 


The CNN model was developed with four convolutional layers, two pooling layers, dense layer and 
dropouts. It uses the ‘relu’ and ‘softmax’ activation function and a learning rate of 0.01. The model was 
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fitted to the training data and the encoded training label and an evaluation carried out on the developed 
model to check for the percentage performance in accuracy and loss. 


Implementation Description of the Model 


During implementation, the test dataset is used with the created model to ascertain the performance of the 
model. This is done with an entirely different dataset or in actual implementation, images capture from a 
life camera feed to check if there is a movement in the railway infrastructure. When the test dataset is 
evaluated, the system shows how many images have movements in it and how many images doesn’t. 


Implementation Analysis and System Testing 


The proposed model for detecting movement in railway infrastructure has been analyzed using the 
convolutional neural network, a machine learning algorithm which is foremost in image or video 
classification data. A total of 20 Images that shows movement and non-movement in railway 
infrastructure were gotten from Google image search for the demonstration and analysis of the proposed 
system. The images were read into the work environment and preprocessed to make them readable and 
usable by the CNN model. The images were resized to 100 x 100 and the summary of the image showed 
that the value of the image lies between o and 1, with 3 channels (meaning that the image is a coloured 
image). The image dataset was split into train and test data with 20% of the dataset used for validation and 
was encoded to the categorical values of ‘0’ and ‘1’ with ‘0’ meaning there is no movement and ‘1’ 
meaning that there is movement on the railway infrastructure. The CNN model created using the Keras- 
model-sequential, it showed a total and trainable parameter of 138,740,514. The model has the epoch size 
of 60 (total number of iteration). 


System Result 


The created model was executed several times to optimize the results obtained since the model keep 
falling short of expectations. The first two results were not very good basically displaying all images as 
not having any movement. After executing the model for the 3" and 4" time, there was a significant 
improvement from the initial results which were record. For the purpose of performance demonstration, 
one of the two different obtained results of the third and fourth run is presented. First of all, few of the 
dataset used to develop and test the model is shown in Figure 2 and the graphical representation of the 
training and test data displayed in figure 3. 


Figure 2: Test Image View of the Proposed System Showing Movement and Non-Movement 
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Figure 2 is the display of tiled images of the test dataset during implementation of the proposed system. 
Four images showing the absence of movement in the railway infrastructure and four images showing 
movements in the railway infrastructure. The dataset was used to train the model and Figure 3 produced 
while the training process was going on. 


E acc E val_acc 


Figure 3: Third Plotted Graphical Representation of the Created CNN Model for the Proposed 
Model for Detection of Movement in Railway Infrastructure System 


From figure 3, we can see that the figure has two graphs on it. The upper graph is the graphical 
representation of the loss (with the blue line designated for actual loss and the green line for validation 
loss) while the lower graph plot for accuracy of the model with the blue line showing the actual accuracy 
and the green line showing the validation accuracy. From the graph, we saw that the accuracy was 
consistently hitting below 70% with little fluctuations but towards the end, there was an increase in 
accuracy. The evaluation result displays that the loss came to 0.4% with an accuracy of 92%. This is 
further shown in the plot history of the model in Figure 4. 
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Figure 4: Third History Graphical Representation of the Created CNN Model for the Proposed 
Model for Detection of Movement in Railway Infrastructure System 


Figure 4 further illustrates that the performance of the CNN model was below expectation. From the 
legend, we see that the red line indicates training while the green line indicate validation with the upper 
graph showing loss and the lower graph showing accuracy within the 60 epoch (or iterations). The 
training line alternates between 50% and 90% while the loss lies slightly below 1. The model has not 
completely classified the provided data and Table 1 and 2 further explains and showcase the shortfalls of 
the model. 


Table 1: Confusion Matrix Training Dataset 


Actual 
Predicted 0 1 
0 6 1 
1 0 


Table 1 is the confusion matrix of the training dataset. It shows the actual and the predicted class of the 
model. Two levels are used to demonstrate the movement and non-movement in the railway 
infrastructure. The class ‘0’ indicates that there is no movement in the image data while the class ‘1’ 
indicates that there is movement in the image data. A total of 6 images that has movement and 6 images 
that doesn’t have movement were used and the confusion matrix has shown that the model during training 
has been able to classify correctly all the non-movement images while it classified 5 images to have 
movement. The table also shows that it predicted one image to have no movement while in actual sense, 
there was movement. This is further shown in the probability distribution table in Table 2. 


© 2022, CAJOTAS, Central Asian Studies, All Rights Reserved 


Copyright (c) 2022 Author (s). This is an open-access article distributed under the terms of Creative Commons 
Attribution License (CC BY).To view a copy of this license, visit https://creativecommons.org/licenses/by/4.0/ 


CENTRAL ASIAN JOURNAL OF THEORETICAL AND APPLIED SCIENCES 


Volume: 03 Issue: 07 | Jul 2022, ISSN: 2660-5317 


Table 4.2: Probability Table for the Trained CNN Model 


SN PROB. OF NO PROB. OF PREDICTED ACTUAL 
MOVEMENT MOVEMENT CLASS CLASS 
1 0.8714516 0.12854840 0 0 
2 0.8885577 0.11144229 0 0 
3 0.8648707 0.13512935 0 0 
4 0.7177815 0.28221846 0 0 
5 0.5249715 0.47502846 0 0 
6 0.9215740 0.07842598 0 0 
7 0.2230344 0.77696562 1 1 
8 0.4521366 0.54786336 1 1 
9 0.3383757 0.66162431 1 1 
10 0.2855883 0.71441174 1 1 
11 0.7012650 0.29873496 0 1 
12 0.4770574 0.52294254 1 1 


From the probability distribution in Table 2, we can see that the probability of No Movement to the 
probability of Movement in the first row shows that the system predicted the image correctly. The actual 
class is ‘0’ and the system predicted ‘0’ showing a 87.1% chance that there is no movement as against 
12.8% chance as seen in the table. The system correctly predicted from Serial Number 1 to 10 and 12, but 
was confused as it miss predicted serial number 11 indicating that there was no movement while in the 
actual sense, there was movement. When the model was used against the test data, the performance was 
very poor as seen in the confusion matrix in Table 3. 


Table 3: Confusion Matrix Test Dataset 


ACTUAL 
PREDICTED 0 1 
0 1 
1 3 2 


The test data comprises of four non-movement images and four movement images. Table 3 shows that the 
model when used with the test data has confuse 2 images that there is no movement whereas, there is 
movement and has confused 3 images that there is movement whereas, there is non-movement. This is 
further shown in the probability distribution table in Table 4. 


Table 4: Probability Table for the Test Dataset with Model 


SN PROB. OF NO PROB. OF PREDICTED ACTUAL 
MOVEMENT MOVEMENT CLASS CLASS 
1 0.6032690 0.3967310 0 0 
2 0.2977493 0.7022507 1 0 
3 0.3902090 0.60979 10 1 0 
4 0.3265038 0.6734962 1 0 
5 0.3734505 0.6265495 1 1 
6 0.5991044 0.4008956 0 1 
7 0.4472865 0.5527135 1 1 
8 0.5718120 0.4281880 0 1 
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From Table 4, we can see that serial numbers 2, 3, 4, 6 and 8 were misclassified when looking at the 
actual and predicted class. The probability distribution showed that as high as 70% and as low as 57% 
were the values used in this misclassification. When evaluated for accuracy and loss, it was found that the 
test data has a loss of 0.82 (82% loss) while the accuracy was 37.5%. After the fourth run, the result was 
not any better hence, the third run was adopted. 


Discussion of Result 


The proposed model for detecting movements in railway infrastructure has been developed using 
Convolutional Neural Network. The system uses the image processing method to achieve this research 
work and these images were acquired through Google image search. The system achieves all the objective 
of the research work as the system was able to identify the targeted movements usually seen in the railway 
infrastructure. A model framework for the detection of movement was created as seen in Figure 1 that 
shows the different modules involved in the detection process. We were able to acquire 20 images that 
show movement and non-movement in the railway infrastructure and they were pre-processed for the 
detection analysis of the proposed system. The images were encoded and fitted to the CNN model and 
detection analysis was obtained. 


In Figure 2, we have the display of the dataset images showing the movements and non-movement in the 
railway infrastructure that was used in the system. In Figure 3, the accuracy consistently hits below 70% 
and later increased which we saw at the evaluation result as 92% was recorded as accuracy with a loss of 
0.4. The effect of the loss was later shown in Table 1; the table showed that the system confused one of 
the images and classified it wrongly. The 6 images in the training set that indicates non-movement were 
correctly classified and 5 out of the 6 images in the training set that indicates movement were also 
classified correctly. One of the images that indicates movement was classified as non-movement as shown 
in Table 2 with a probability of 70.1% misclassification. After the model was used against the test dataset, 
the model produced a very poor result as more than half of the test data were misclassified as shown 
Table 3 and Table 4. In Table 4, the probability distribution showed that as high as 70% and as low as 
57% were the values of this misclassification. When evaluated for accuracy and loss, it was found that the 
test data has a loss of 0.82 (82% loss) while the accuracy was 37.5%. 


Conclusion 


We therefore conclude that, the model for detecting movement in railway infrastructure using 
Convolutional Neural Network in image processing has performed poorly in classifying and detecting 
movement. The performance accuracy of the training data came to 92% with 0.4 % loss when evaluated 
and the probability distribution on the predicted class carried out showed that one out of the 12 image 
dataset used for training was misclassified. But, when the CNN model was used to fit with the test data, it 
demonstrated a very poor result with a percentage accuracy of 37.5% with a huge loss of 0.82% which 
was very clear when viewed through the probability distribution table of the test data. This model can be 
improved upon to achieve optimal classification and detection. The images used and adjustment to the 
model can increase the performance of the model’s accuracy and reduce the loss drastically. 
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